Method and apparatus for initiating routing messages in a communication network

ABSTRACT

Switches within a telecommunications network exchange so-called available bandwidth messages, each of which advertises how much bandwidth remains unassigned on a respective link. The network is of a type in which circuits are provisioned with various predefined numbers of time slots (equivalent to bandwidth). The sending of an available bandwidth message for a given link is triggered by a change in the number of time slots available on that link if that change results in a change in the number of circuit bandwidths that can be accommodated by that link for a newly provisioned circuit.

CROSS-REFERENCE TO RELATED APPLICATION

This is a continuation of Ser. No. 10/786,802, filed Feb. 25, 2004.

BACKGROUND

The present invention relates generally to the generation of routingmessages in telecommunications networks.

Telecommunications networks comprise switches interconnected bycommunication links. A circuit—which may be customer-requested or may beprovisioned by the network operator in order to support, for example,switched traffic operations—is established by setting up a path thatcomprises a sequence of two or more switches and links connecting them.Routing protocols, such as OSPF (Open Shortest Path First), MPLS(Multi-Protocol Label Switching) and PNNI (Private Network to NetworkInterface), are utilized by the switches to send routing messages amongthemselves. These routing messages are used to, for example, disseminatenetwork state information throughout the network. So-called signalingmessages are communicated among the switches to allocate the bandwidthon particular links in order to provision the circuits, using thedisseminated network state information.

The present invention concerns itself with routing messages transmittedamong the switches of the network that advertise how much bandwidthremains unassigned on respective links. A message containing suchinformation is referred to herein as an “available bandwidth message.”Upon receipt of a request to provision a new circuit of a certainbandwidth between an originating switch and a terminal switch, theoriginating switch launches a setup message along a particular paththrough the network to the destination switch. Bandwidth availabilityinformation gleaned from the most recently received available bandwidthmessages is used to select a path having sufficient unassigned bandwidthon all of its links to accommodate the requested circuit. As each switchreceives the setup message, it allocates the requested amount ofbandwidth to the circuit on a link extending to the next switch alongthe path

A design trade-off is involved in the generation and transmission ofavailable bandwidth messages. It is desirable that they be sentrelatively frequently. The greater the time interval between thetransmission of available bandwidth messages, the more likely it is thatthere will be insufficient bandwidth on one or more links identified fora circuit being set up because some or all of the advertised bandwidthmay have been allocated to other circuit(s) in the interim. In this casethe circuit must be “cranked back” to the originating switch from thepoint where the attempted allocation has failed, meaning that acrankback message is sent back along the path in question, causing thebandwidth allocated to the circuit to be de-allocated, link-by-link, allthe way back to the originating switch. The latter must then identify adifferent path through the network that (hopefully) does have enoughbandwidth on each of its links to accommodate the request, and theprocess begins again.

As will be appreciated, the necessity of cranking back large numbers ofcircuits is undesirable in that it represents a waste of networkresources and delays fulfillment of the customer's circuit request. Thenumber of crankbanks resulting from attempted circuit setups usingoutdated available-bandwidth information can be reduced by sendingavailable bandwidth messages more frequently. Indeed, it might bethought that the ideal situation would be to transmit an availablebandwidth message for a link whenever the available bandwidth on thatlink changes. That approach, however, engenders its own problems in thatit increases the amount of routing traffic in the network, therebyrequiring that the network be designed with enough resources toaccommodate the increased traffic level, including the resources neededto formulate the messages, transmit them, and assimilate the informationthat they contain.

Even if the network has sufficient resources to handle normal circuitsetup operations, it could become unduly congested when a large numberof circuits need to be set up over a short period of time, such asduring a restoration of service after a cable cut or other catastrophicnetwork event has occurred, thereby impairing the network's ability tocommunicate all kinds of signaling messages needed to carry out therestoration process and thereby impairing the network's ability torapidly reroute its circuits when such events occur.

The magnitude of the aforementioned problem can be more fullyappreciated by considering the way in which available bandwidth messagesare transmitted in a system that uses, for example, the above-mentionedPNNI routing protocol. The switches in the network can exist at variouslevels of the PNNI routing hierarchy. A node in the hierarchy can be asingle switch or can be a number of switches assigned to a peer group,wherein each node in the peer group exchanges information with othernodes in the peer group so that all nodes maintain an identical view ofthe peer group. A so-called peer group leader communicates summaryinformation relating to its own peer group, including the state of itslinks, to switches outside of the peer group, including to other peergroup leaders.

The nodes within a peer group communicate routing information to oneanother using a type of message referred to as a PNNI Topology StateElement, or PTSE. One particular type of PTSE transmitted by a switch iscalled a link PTSE, which contains information about a particular link.This information includes the amount of bandwidth available on the link.The link PTSE thus functions as the available bandwidth message withinthe PNNI protocol.

Nodes within a peer group communicate link PTSEs among themselves usinga process referred to as “flooding.” Flooding is the reliable hop-by-hoppropagation of link PTSEs throughout a peer group to ensure that everynode gets every link PTSE. Every node must send its own(self-originated) link PTSEs out on all its interfaces or links from thenode. Every node, upon receiving a link PTSE from another node, isexpected to send a copy out on all its interfaces or links except,perhaps, the one that goes back to the node from which the link PTSE wasreceived. Some of the switches could have more than fifty links to otherneighboring switches (some of which may be in other peer groups),although there may be several parallel links to the same neighbor. Manyswitches are smart enough to recognize that if there are multiple linksto the same neighbor, then they flood available bandwidth messages ononly one of these links. Still, the number of neighbors could be large,often greater than fifteen. This means that a switch with an adjacency(number of neighbors) of say, sixteen, could see as many as fifteenduplicates of each link PTSE due to the way flooding works. A duplicatemessage is discarded but only after a fair amount of processing is doneon it. This includes time to process all lower layer encapsulations, aswell as the link PTSE message header, and a routing database lookup.This can result in a lot of wasted CPU cycles and can create a high loadon the CPU of the switch, often at times (like a large restorationevent) when the CPU can least afford wasted cycles. The duplicatemessages can also clog queues and result in message discards if thequeues fill up.

During a large failure/restoration event, hundreds of links can beaffected and the available bandwidth on a link can change rapidly as thefailed connections release previously allocated bandwidth and then asrestoring connections reassigns available bandwidth. This can result inhundreds, and sometimes thousands, of link PTSEs being flooded to a nodein a very short time.

The above problems are ameliorated in the prior art to some extent byimposing certain constraints on how often and under what constraints anavailable bandwidth message is sent. In particular, prior artarrangements send an available bandwidth message for a link only whenthe amount of bandwidth available on that link changes (increases ordecreases) by a predetermined percentage amount and/or by apredetermined absolute amount. Indeed, by adjusting how great thatchange needs to be before a message is sent out, the network is able toachieve a workable balance between the timeliness of thebandwidth-availability data and the number of available bandwidthmessages launched into the system per unit time. Moreover, the prior artimposes certain time restrictions on the sending of the availablebandwidth messages, e.g., the sending by a node or switch of no morethan one link PTSE per second for a given link, no matter how many timesthe available bandwidth of that link changes during that time period.

SUMMARY OF THE INVENTION

The present invention provides a way of yet further reducing the numberof available bandwidth messages, such as link PTSEs, that are launchedinto the network per unit time without causing any significant increasein crankbanks.

The invention takes advantage of the fact that, in many systems,circuits are provisioned in discrete bandwidth amounts. We have thusrealized that the fact that the available bandwidth on a link haschanged is not a useful piece of information in such systems unless thenumber of different circuit bandwidths that are available for newlyprovisioned circuits has changed. That is, the amount of availablebandwidth change for the link is such as to either a) allow for the nextlevel of circuit bandwidth to be accommodated or b) make a previouslyavailable circuit bandwidth no longer available. Implementationally,this mode of operation can be achieved by sending an available bandwidthmessage only when the available bandwidth has either a) increased fromits previous value to a value at least equal to the next higher circuitbandwidth or b) has decreased from its previous value to a value that islower than the next lower circuit bandwidth.

For example, the circuits in optical transport networks are typicallySTS-N (Synchronous Transport Signal level N) circuits having N timeslots, where, in current or planned future systems, N is 1, 3, 12, 24,48 or 192 (the number of time slots used by a circuit being a measure ofits bandwidth). These circuits are referred to as STS-1, STS-3, STS-12,STS-24, STS-48 and STS-192 circuits. In order to route an STS-N circuitover a particular link, it is enough to know whether N slots areavailable on the link or not. For example, it is of no importance thatthe available bandwidth on a link has changed from 15 slots to 21 slots,because prior to the change that link could have accommodated a newSTS-1, STS-3 or STS-12 circuit and after the change, that link can stillonly accommodate those same three circuit bandwidths. Rather, what isimportant to know is whether the available bandwidth on this link haseither a) increased to at least 24, because a new STS-24 circuit couldnow be accommodated in addition to the three STS-1, STS-3 or STS-12circuit bandwidths, or b) has decreased to less than 12, because now anSTS-12 circuit can no longer be accommodated, leaving only the STS-1 andSTS-3 circuit bandwidths. Thus the criterion to be used in determiningwhether a change in available bandwidth should be advertised is whetherthe new available number of time slots either a) increased from itsprevious value to a value at least equal to the next higher one of thetime slot amounts 1, 3, 12, 24, 48 or 192, or b) has decreased from itsprevious value to a value that is lower than the next lower one of thosetime slot amounts.

It is also possible to define the invention as causing an availablebandwidth message to be sent if the amount of available bandwidth hascrossed any one of a plurality of thresholds.

Particular embodiments of the invention may also incorporate variousaspects of prior art practice, such as the limitation that a switch willtransmit no more than one available bandwidth message relative to agiven link within a given time period, such as one second.

Another advantage of the invention is that the reduction in the numberof available bandwidth messages that are generated—particularly duringrestoration—will allow the scaling of the network to a larger sizewithout requiring a concomitant increase in the amount of resourcesneeded for the transmission of available bandwidth messages.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is an illustrative network in which the present invention isimplemented;

FIG. 2 shows a different view of the network of FIG. 1, helpful inillustrating certain aspects of the PNNI routing protocol;

FIG. 3 is a block diagram of one of the switches in the network of FIG.1; and

FIG. 4 is a flowchart of the basic functions carried out by a switch ofFIG. 3 in implementing the principles of the invention—specifically fordetermining when a available bandwidth message should be transmitted.

DETAILED DESCRIPTION

FIG. 1 shows an illustrative network 10 in which the present inventionis implemented. Network 10 includes a plurality of switches 101-106 anda plurality of point-to-point communication links 201-207. Links 201-207can be OCN optical links such as OC-48, OC-12, OC-3 and/or DS-3communication links. Although not shown, there can be multiple linksbetween a pair of switches. Each link 201-207 is typicallybi-directional and has potentially different characteristics in eachdirection. For example, the various links could have respectivedifferent bandwidths and administrative weight in each direction. It isassumed in this example that all the links have the same characteristicsin both directions. As is well-known, multiple links can also be groupedinto an “aggregated link.”

Switches 101-106 may be, for example, optical switches, ATM(Asynchronous Transfer Mode) switches, FR (Frame Relay) switches orIP/MPLS routers. The switches can automatically discover the network andset up circuits using known link-state routing and signaling protocols.

Circuits established between a pair of switches can include one or moreintermediate switches. The service route of the circuit is the set oflinks and switches on which it is set up. FIG. 1 illustrates aparticular circuit 301 that has been set up through network 10 tointerconnect end systems 8 and 9. As can be seen, circuit 301 includesswitches 101, 104, 105 and 103 and links 205, 206 and 207, as well asthe access network connections between end systems 8 and 9 and switches101 and 103, respectively.

It is assumed in the present illustrative embodiment that routingmessages—which provide information about network topology, includingavailable bandwidth on the various links—are communicated among thevarious switches using the PNNI protocol. FIG. 2 shows another view ofnetwork 10 helpful in explaining certain aspects of the PNNI protocol.In particular, nodes in the PNNI hierarchy comprise individual switchesand/or groups of switches, the latter being referred to as peer groups.FIG. 2 depicts each one of switches 101 through 106 as a node and showsthat network 10 further includes switches 107 through 109 that are notshown in FIG. 1. Network 10 may, of course, include any desired numberof switches. FIG. 2 also shows links 201 through 207, as well as links208 through 213 not shown in FIG. 1.

Switches 101, 106, 107 and 108 constitute a peer group for PNNIpurposes, denoted as peer group 220. As previously noted, link PTSEs aremessages transmitted among the switches within a peer group. In additionto being in the same peer group as switch 101, switches 106, 107 and 108are all neighbors of switch 101 since each is connected to switch 101 bya link. Switches 107 and 108, although in the same peer group, are notneighbors since there is no link connecting them. Switches also haveneighbors that are not in the same peer group. For example, switch 104is a neighbor of switch 101. Although only one peer group is indicatedin the FIG., a typical network will include many peer groups. Theconcept of “neighbor” is important in that even though a switch (otherthan the peer group leader) may only send and receive routing messageswith the members of its peer group, a switch sends and receivessignaling messages with its neighbors during, for example, call setup,even if a neighbor does not belong to its peer group.

Signaling, routing and other messages are communicated among the variousswitches over channels carried by the same links, i.e., links 201through 213, that carry the customer traffic. For example, a dedicatedchannel could be set aside on each link for this purpose or inbandsignaling could be used or a channel within the SONET overhead could beused. Although not envisioned for the present embodiment, the switchescould, alternatively, communicate signaling and routing messages over atotally separate network similar to the conventional type of SS7network, or they might communicate over a separate IP network.

Arrows 225 in FIG. 2 represent respective signaling or routingmessages—which could be, for example, link PTSEs—sent by switch 106 toswitch 107 and vice versa.

Network 10 allocates circuits in discrete bandwidth amounts. Moreparticularly, network 10 is illustratively an optical transport networkin which the provisioned circuits are STS-N circuits, such as STS-1,STS-3, STS-12, STS-24, STS-48 and STS-192 circuits, which require 1, 3,12, 24, 48 and 192 time slots (equivalent to bandwidth), respectively.In order to route an STS-N circuit over a particular link, it is enoughto know whether N slots are available on the link or not. Thus, inaccordance with the present invention, we have recognized that a changein available bandwidth is sufficient to cause a new available bandwidthmessage, i.e., link PTSE, to be transmitted only if that change inbandwidth changes the number of circuit bandwidths that are available onthat link for a newly provisioned circuit. For example, as notedearlier, a link having 15 available time slots can accommodate threecircuit bandwidths for a newly provisioned circuit—an STS-1, an STS-3 oran STS-12 circuit—and after the change, it can still only accommodatethose three circuit bandwidths. Thus the fact that the availablebandwidth has changed from, say, 15 time slots to 21 time slots is nothelpful information and such a change will not trigger the sending of anew link PTSE. On the other hand, a change from 15 time slots to 24 timeslots is important to know because four circuit bandwidths can now beaccommodated—STS-1, STS-3, STS-12 and STS-24. Similarly a change from 15time slots to 10 time slots is important to know because only twocircuit bandwidths can now be accommodated—STS-1 and STS-3.Implementationally, the criterion to be used in determining whether achange in available bandwidth should be advertised is whether the newavailable number of time slots has become either a) at least as great asor b) less than (in this example) the set of thresholds 1, 3, 12, 24, 48or 192.

It is also possible to define the invention as causing an availablebandwidth message to be sent if the amount of available bandwidth hascrossed any one of a plurality of thresholds. With such a definition, wemust take account of the fact that an upward change in the amount ofavailable bandwidth is important if the new amount at least equals thenext higher bandwidth (or time slot) amount. By contrast, a downwardchange in the amount of available bandwidth is important if the newamount crosses below the next lower bandwidth (or time slot) amount.However, as long as we understand the aforementioned thresholds to eachbe slightly less than one of the discrete circuit bandwidth amounts, itis indeed possible to define the invention in the way just suggested.For example, if the aforementioned set of thresholds is taken to be 0.5,2.5, 11.5, 23.5, 47.5 and 191.5 (that is, 0.5 less than the standardtime slot values 1, 3, 12, 24, 48 and 192), then it can be said that theinvention causes an available bandwidth message to be sent whenever theamount of available bandwidth crosses any one of those thresholds ineither the up or down direction. It can thus be said, in general, thatthe aforementioned thresholds are each a function of one of the discretebandwidth amounts.

It should be noted in this regard that although the invention canactually be implemented by doing threshold comparisons of this type,other ways of implementing the invention are possible. Such other waysof implementing the invention may nonetheless be seen as inherentlymeeting this threshold-based definition of the invention. That is, ifthe available bandwidth of a link changes from 3 time slots to 12 timeslots and an available bandwidth message is transmitted as a result ofthat change, one can say that the available bandwidth message wastransmitted in response to the available bandwidth having crossed athreshold, e.g., 11.5, even if the determination that that change from 3to 12 occurred did not involve comparing 3 and/or 12 with 11.5. That is,the value 11.5 was, indeed, crossed when the change happened.

FIG. 3 is a generalized block diagram of switch 101, taken as exemplary.Switch 101 can be characterized as having two basic types ofcomponents—processing and other circuitry 121 and memory 122. Withinmemory 122 is a table 1223 containing a list of the aforementionedbandwidth thresholds. Memory 122 also contains programs 1224 that areexecuted by circuitry 121 to carry out the various functions andfunctionalities of the switch, including the transmission of availablebandwidth messages, illustratively link PTSEs, pursuant to theprinciples of the invention.

FIG. 4 is flowchart of the basic functions carried out by switch 101 inimplementing the decision as to when a link PTSE should be sent,pursuant to the principles of the present invention—specifically fordetermining when an available bandwidth message should be transmitted.

The process begins at 401 in response to the switch having allocated orreleased bandwidth on one of its associated links. It is then determinedat 403 whether the number of available circuit bandwidths has changed.This is equivalent in this embodiment to determining whether the numberof time slots available on the link has either a) increased from itsprevious value to a value at least equal to the next higher circuitbandwidth, or threshold, as defined in table 1223 or b) has decreasedfrom its previous value to a value that is lower than the next lowercircuit bandwidth. Referring again to the above example, if theavailable bandwidth prior to the change was 15 time slots, step 403determines whether the number of times slots now available on the linkis at least equal to 24 or is lower than 12.

If the answer at 403 is “yes,” it is then determined at 404 whether theswitch has sent out a link PTSE for this link within the previous onesecond because, as noted above, it is desirable to impose a minimum timeperiod between the sending of successive PTSEs. A check is thereforemade at 404. If a link PTSE indicating the new available bandwidth wasnot sent within the previous one second, then a link PTSE is sent at407. Otherwise, the process waits at 408 until that one second hasexpired and then the link PTSE is sent at 407. If the amount ofavailable bandwidth for this link changes during the wait, then the linkPTSE that is ultimately sent indicates the latest value.

Although not shown in the FIG., a switch will send a link PTSE for eachlink periodically, e.g., once every half hour, whether or not theavailable bandwidth on that link has changed by any particular amount,if any. This helps ensure that the network is operating with correctrouting data.

The foregoing merely illustrates the principles of the invention.

For example, the present invention has been described with each switchhaving a list of thresholds that it applies to all its links. Theinvention also applies to different switches having different lists ofthresholds, as well as to a switch having multiple lists of thresholdsand applying different lists to different links.

The present invention is applicable to other MPLS-based IP (InternetProtocol) networks and the traditional ATM and Frame Relay (FR) networksas well. The present invention can also be used with anytelecommunications network with switches capable of establishingcircuits—for example, Frame Relay switches, ATM switches, IP/MPLSrouters, optical switches, digital and optical cross-connects, to name afew.

It should be understood that the present invention can be employed inrouting protocols in general. Furthermore, the present invention can beemployed in systems using routing protocols that are compliant withvarious routing standards and their variants, including but not limitedto the OSPF routing protocol.

It will thus be appreciated that although the principles of the presentinvention have been illustrated in conjunction with a specificembodiment, those skilled in the art will be able to devise manyalternatives, modifications and variations that embody those principlesand thus are within their spirit and scope.

1. A method for use in a communication network that includes a pluralityof switches, the method comprising advertising an amount of availablebandwidth for a link in response to said available bandwidth havingcrossed any one of a plurality of fixed bandwidth thresholds, whereinsaid communication network allocates bandwidth to circuits establishedover said link in discrete bandwidth amounts, wherein said plurality ofbandwidth thresholds are each a function of said discrete bandwidthamounts, and wherein said advertising comprises transmitting, by one ofsaid switches to which said link is connected, an available bandwidthmessage to at least another one of said switches.
 2. The method of claim1 wherein said plurality of bandwidth thresholds are each apredetermined amount smaller than a respective one of said discretebandwidth amounts.
 3. The method of claim 1 wherein individual circuitsset up over said link each utilize a respective number of time slots,and wherein each of said discrete bandwidth amounts corresponds to arespective number of said time slots.
 4. The method of claim 3 whereineach of said individual circuits is an STS-N circuit having N timeslots, where N is a value selected for each circuit from among apredefined set of values.
 5. The method of claim 3 wherein saidavailable bandwidth message is a link PTSE message defined by the PNNIprotocol.
 6. A method for use in a communication network comprising aplurality of switches interconnected by a plurality of links, the methodcomprising setting up circuits through said network, each circuit beingset up over a path that includes two or more of said switches and one ormore of said links and each circuit having a particular amount ofbandwidth selected from a plurality of predetermined circuit bandwidths,and responsive to a request to set up through said network an additionalcircuit having a desired amount of bandwidth, identifying a path throughsaid network that includes links each having at least that amount ofavailable bandwidth, wherein it is determined how much bandwidth eachlink has available from available bandwidth messages transmitted withinsaid network, each of the available bandwidth messages indicating anamount of available bandwidth for a respective link, each of at leastones of said available bandwidth messages being transmitted responsiveto a determination that the available bandwidth of a particular link haseither a) increased from a previous value to a value at least equal tothe next higher one of said predetermined circuit bandwidths or b) hasdecreased from a previous value to a value that is lower than the nextlower one of said predetermined circuit bandwidths, and wherein saidavailable bandwidth messages are transmitted by at least ones of saidswitches to others of said switches.
 7. The method of claim 6 whereinsaid circuits each utilize a respective number of time slots, andwherein each of said predetermined circuit bandwidths corresponds to arespective number of said time slots.
 8. The method of claim 7 whereineach of said circuits is an STS-N circuit having N time slots, where Nis a value selected for each circuit from among a predefined set ofvalues.
 9. The method of claim 6 wherein said network utilizes the PNNIrouting protocol and wherein said available bandwidth messages are linkPTSE messages defined by said PNNI protocol.
 10. A method for use in atelecommunications network in which circuits are established over pathswithin said network, each said path comprising at least two switchesinterconnected by at least one link, each of said circuits having aprovisioned amount of bandwidth selected from a predetermined set ofcircuit bandwidths, said network being of a type in which availablebandwidth messages transmitted within said network indicate an amount ofbandwidth currently available on an associated one of said links toprovision additional circuits that include that link, the methodcomprising initiating the transmission of an individual one of saidavailable bandwidth messages in response to a change in the number ofsaid circuit bandwidths that are available on the associated link forthe provisioning of new circuits.
 11. The invention of claim 10 whereinavailable bandwidth messages associated with a particular link aretransmitted by at least one switch to which that link is connected. 12.The invention of claim 11 wherein said at least one switch transmitssaid available bandwidth messages associated with a particular link toother switches of said network.
 13. The invention of claim 11 whereinsaid at least one switch transmits said available bandwidth messagesassociated with a particular link to other switches of said network, butonly if it has not done so within a predetermined period of time sinceit last transmitted an available bandwidth message associated with saidparticular link.
 14. A switch for use in a telecommunications network inwhich circuits are established over paths within said network, each saidpath comprising at least two switches interconnected by at least onelink, each of said circuits having a provisioned amount of bandwidthselected from a predetermined set of circuit bandwidths, said networkbeing of a type in which available bandwidth messages transmitted withinsaid network indicate an amount of bandwidth currently available on anassociated one of said links to provision additional circuits thatinclude that link, the switch comprising circuitry including processingcircuitry, and a memory having stored therein programs that, whenexecuted by said circuitry, initiate the transmission of an individualone of said available bandwidth messages in response to a change in thenumber of said circuit bandwidths that are available on the associatedlink for the provisioning of new circuits.
 15. The invention of claim 14wherein said programs, when executed by said circuitry, initiate thetransmission of said individual one of said available bandwidth messagesassociated with a particular link only if an available bandwidthmessages associated with said particular link has not been transmittedwithin a predetermined period of time since an available bandwidthmessage associated with said particular link was last transmitted.